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Abstract 


A  discrete  event  simulation  was  developed  combining  the  Optical  Tracking  Network  (TeleTrak)  and  Satellite  Position 
Attained  by  RF-Keyed  Tracking  (SPARK)  system  into  one  system  to  track,  image,  and  identify  space  objects  as  they 
pass  through  the  space  fence  to  increase  space  situational  awareness  for  the  United  States  Air  Force.  The  objectives 
for  this  study  are  threefold:  model  a  "to-be"  architecture  for  a  combined  TeleTrak  and  SPARK  system  to  develop 
system  requirements,  determine  if  an  optimal  position  for  the  telescope  exists  to  return  to  while  waiting  for  the  next 
collect,  and  demonstrate  knowledge  and  understanding  of  DOE  and  simulation  principles  and  techniques.  The  paper 
explores  the  uses  and  pitfalls  of  modeling  and  simulation  with  this  case  study.  While  a  working  model  remained 
elusive,  several  important  observations  emerged.  Six  recommendations  are  given  to  help  others  become  more 
successful  in  a  modeling  and  simulation  environment  along  with  ideas  to  pursue  in  additional  research. 
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L  Introduction 

This  paper  analyzes  the  design  of  experiments  (DOE)  and  discrete  event  simulation  for  the  Air  Force 
Institute  Technology  (AFIT)  combined  Optical  Tracking  Network  (TeleTrak)  and  Satellite  Position 
Attained  by  RF-Keyed  Tracking  (SPARK)  system.  The  objectives  for  this  study  are:  (1)  model  a  "to-be" 
architecture  for  a  combined  T eleT rak  and  SPARK  system  to  develop  system  requirements,  (2)  determine 
if  an  optimal  position  for  the  telescope  exists  to  return  to  while  waiting  for  the  next  collect,  and  (3) 
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demonstrate  knowledge  and  understanding  of  DOE  and  simulation  principles  and  techniques. 

The  A  FIT'S  TeleTrak  goal  is  to  use  commercial  telescopes  to  determine  orbit  estimation  and  collect 
object  images  of  low  earth  orbit  (LEO)  space  objects.  If  research  demonstrates  high  reliability  and 
usability  while  providing  lower  cost,  theTeleT rak  system  could  be  used  to  support  the  space  surveillance 
network  in  detecting,  tracking,  identifying  and  cataloging  all  manmade  objects  orbiting  the  Earth. 
Currently,  the  system  uses  two  ten-inch  and  one  sixteen-inch  R itchey-C hreti en  telescopes  geographically 
separated  from  each  other  coupled  with  M  ATLAB  ®  software  to  track  and  image  an  observed  LEO  space 
object1.  M  eanwhile,  the  AFIT's  SPARK  system  goal  is  to  use  commercially  available  antennas  to  collect 
the  signal  from  the  Air  Force  Space  Surveillance  System  (AFSSS)  as  a  space  object  crosses  the  space 
fence  in  order  to  determine  orbit  estimation  and  tracking  information2.  The  two  systems  are  currently 
only  loosely  tied  together  with  analysis  being  done  on  the  T eleT rak  system  to  enhance  image  clarity  and 
tracking  reliability,  and  on  the  SPARK  system  to  develop  orbit  propagation  solution  for  tracking. 

The  "to-be"  architecture  scenario  is  where  the  systems  working  jointly  and  autonomously  together 
track,  image,  and  determine  the  orbit  for  the  space  object.  The  scenario  begins  with  a  space  object 
crossing  the  space  fence  which  the  SPARK  system  picks  up.  Determination  is  made  on  the  object  if  it 
matches  an  already  defined  orbit  for  a  satellite  orbiting  the  Earth  described  in  orbital  elements  by  a 
NORAD  two  line  element  (TLE)  set.  If  so  then  a  determination  is  made  if  it  is  a  planned  collection  object 
or  an  object  of  opportunity.  If  object  is  not  matched  then  it  is  classified  as  unidentified  and  tracked  with  a 
higher  priority;  the  TeleTrak  team  would  take  steps  to  resolve  unidentified  objects  in  post- processing  of 
data.  Based  on  the  determinations  an  object’s  priority  and  number  of  telescopes  required  for  tracking  are 
assigned  according  to  table  1. 

T able  1:  Space  Object  Priority  and  Telescopes  Required  when  T racking 


Priority 

N  umber  of  T elescopes  for  T racking 

U  nidentified  Object  1 

2 

Planned  Object  2 

2 

Object  of  Opportunity  3 

1 

The  SPARK  system  then  provides  an  azimuth  and  elevation  of  where  to  acquire  the  space  object  to  the 
T  eleT  rak  system.  TheTeletrak  system  stops  processing  any  tracking  if  a  higher  priority  task  comes  in,  if 
not,  then  checks  to  see  if  a  telescope  is  available.  W  hen  a  telescope  is  available  then  the  system  slews  to 
the  azimuth  and  elevation  provided  by  SPARK  and  begins  target  acquisition,  tracking  and  image 
collection.  Image  collection  is  accomplished  through  a  Webcam  or  StellaCam  inserted  in  the  eyepiece 
and  data  is  transmitted  to  a  computer.  Tracking  and  imaging  continues  till  the  object  is  approximately 
fifteen  degrees  from  the  horizon  due  to  light  pollution  around  the  telescopes  at  a  lower  elevation.  Once 
complete  with  the  collect  the  telescopes  slew  to  the  next  target  or  return  to  a  pre-determined  location. 

Z  Methodology 

A  twelve  step  process  proposed  by  J .  Banks  and  others  was  used  in  developing  the  simulation  model. 
This  method  provided  a  structured  analytical  approach  to  be  applied  during  development.  The  twelve 
steps  are3: 

(1)  Problem  Formulation,  (2)  Setting  of  Objectives  and  Overall  Project  Plan,  (3)  Model 
Conceptualization,  (4)  Data  Collection,  (5)  Model  Translation,  (6)  Verified?,  (7)  Validated?,  (8) 
Experimental  Design,  (9)  Production  Runs  and  Analysis,  (10)  More  runs?,  (11)  Documentation  and 
Reporting,  (12)  Implementation. 

A  seven  step  process  as  described  by  D.  C.  Montgomery  is  used  to  further  enhance  the  design  of 
experiment  aspects  in  the  simulation  creation  process,  mainly  in  the  areas  of  Data  Collection  and 
Experimental  Design  steps,  but  also  in  the  Problem  Formulation  and  Setting  of  Objectives  and  Overall 
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Project  Plan.  The  seven  steps  are4: 

(1)  Recognition  of  and  Statement  of  the  Problem,  (2)  Selection  of  the  Response  Variable,  (3)  Choice 
of  Factors,  Levels,  and  Range,  (4)  Choice  of  Experimental  Design,  (5)  Performing  the  Experiment,  (6) 
Statistical  Analysis  of  Data,  (7)  Conclusion  and  Recommendations. 

The  steps  (l)-(8)  for  the  twelve  step  simulation  process  and  steps  (l)-(4)  in  the  seven  step  DOE 
process  relate  to  actions  and  analysis  that  are  accomplished  during  a  Pre-Experi mental  phase  while  the 
remaining  steps  will  be  defined  as  Experimental/Post-Experimental  Phase.  This  allows  for  discussions  on 
what  has  occurred  so  far  in  the  study  and  what  actions  will  occur  after  additional  work  is  accomplished. 

3.  Pre-Experimental  Phase 

3.1.  Problem  Formulation 

The  problem  is  to  model  a  combined  SPARK  and  TeleTrak  system  to  allow  for  identification  of 
possible  system  specifications,  and  to  determine  the  optimal  telescope  azimuth  and  elevation  return  point 
after  a  collection  to  facilitate  the  acquisition  of  the  next  space  object  to  track  and  image.  Possible  system 
specifications  are:  identify  the  maximum  length  of  time  the  SPARK  system  has  to  compute  an  azimuth 
and  elevation  initial  condition  to  pass  to  the  TeleTrak  system,  and  the  minimum  view  time  an  item  would 
need  to  have  to  attempt  tracking  of  that  object  if  the  T eleT rak  system  was  already  tracking  an  object  and 
for  it  to  be  a  benefit. 

3.2.  Setting  of  Objectives  and  Overall  Project  Plan 

The  objectives  are  to  produce  analysis  in  order  to  develop  statements  of  system  specifications.  In 
order  to  achieve  the  objective  several  tasks  were  needed  to  be  accomplished  over  a  period  of  ten  weeks, 
and  they  are  broken  out  in  the  overall  project  plan.  The  first  five  weeks  were  to  be  focused  on  learning 
about  design  of  experiments  and  using  the  simulation  software  EXTENDSIM  8™.  Selection  of 
simulation  software  was  based  on  prior  use,  and  its  robustness  to  handle  several  different  simulation 
methods.  In  the  sixth  week,  determine  if  the  simulation  is  appropriate  to  achieve  the  objectives  and 
formalize  the  problem  statement  with  stakeholders.  In  the  seventh  week,  develop  a  simple  representation 
of  the  system  to  use  as  a  base  to  grow  the  model  in  complexity  as  additional  work  or  insight  into  the 
system  is  obtained.  The  simple  representation  would  consist  of  random  space  object  crossing  the  space 
fence,  the  SPARK  system  developing  azimuth  and  elevation  for  TeleTrak  system  to  begin  target 
acquisition,  the  Teletrak  system  begins  tracking  and  imaging  object  using  two  scopes  geographically 
located  away  from  each  other.  In  the  eighth  week,  expand  the  model  to  allow  ability  for  scopes  to 
independently  track  objects  of  opportunity  and  revert  to  using  two  scopes  on  one  object  for  planned  or 
unidentified  objects  passing  through  the  space  fence.  Also,  during  the  week  develop  a  function  to  model 
the  azimuth  and  elevation  of  the  scope  when  an  object  exits  or  is  interrupted,  so  calculations  can  be  made 
on  the  amount  of  time  required  to  slew  the  telescope  to  the  new  object.  In  the  ninth  week  add  the 
capability  to  use  Two  Line  Element  sets  (TLEs)  from  a  database  or  accomplish  data  collection  to  allow 
modeling  which  would  enhance  object  generation  to  more  accurately  reflect  real  world  conditions. 
T owards  the  end  of  the  ninth  week  accomplish  verification  and  validation  of  model  using  factorial  design 
if  needed,  in  the  tenth  week  accomplish  simulation  runs,  then  use  a  M  onte  Carlo  analysis  to  determine 
azimuth  and  elevation  optimal  locations  and  system  requirements  for  minimum  initial  conditions  of  object 
to  view  given  time  required  to  slew  scope,  and  finalize  simulation  work  by  writing  a  report. 

Before  going  into  model  conceptualization  it  is  beneficial  to  discuss  the  three  types  of  simulations 
available  (continuous,  discrete-event,  and  discrete- rate)  and  the  reasoning  behind  selecting  discrete-event 
for  modeling.  The  different  methodologies5  are:  In  a  continuous  model  the  time  step  is  evenly  spaced 
from  each  other,  and  interactions  of  the  model  occur  with  changes  in  time.  In  a  discrete  event  simulation 
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the  system  changes  only  when  an  event  occurs,  and  is  not  dependant  on  time.  Finally,  in  a  "discrete  rate 
simulation  ...  [the  discrete  event  and  continuous  models  merge  to] ...  simulate  the  flow  of  stuff  rather  than 
items;  like  discrete  event  models  they  recalculate  rates  and  values  whenever  events  occur5."  A  discrete 
event  simulation  was  selected,  because  the  telescopes  would  only  be  looking  at  a  single  item  at  a  time,  the 
system  would  change  whenever  an  object  crossed  the  space  fence  (event)  or  begins  tracking  with  the 
telescope(s)  (event).  Also,  the  slew  times  could  be  approximated  in  the  model  as  an  activity  with  a 
processing  time,  and  flight  times  could  be  simulated  by  a  queue  that  releases  objects  if  they  have  been 
waiting  for  awhile. 

3.3.  Model  Conceptualization 

Currently  the  SPARK  system  is  able  to  identify  where  and  when  an  object  crossed  the  space  fence, 
however  orbit  propagation  is  not  possible  as  of  yet.  The  model  assumes  that  when  the  system  is 
implemented  this  issue  has  been  resolved  and  the  determination  of  the  object  against  the  TLEs  happens  in 
an  instantaneous  manner.  Another  assumption  is  with  the  reliability  of  theTeleTrak  system.  The  model 
assumes  100%  reliability  for  the  system  on  acquiring  and  tracking  the  desired  object  autonomously,  but 
currently  the  system  requires  human  input  for  minor  corrections  to  have  reliability  in  tracking  an  object. 
A  nother  assumption  is  that  any  and  all  objects  would  be  able  to  be  viewed  by  the  telescopes,  which  may 
not  be  the  case  depending  on  the  size  and  brightness  level  of  the  object. 

The  length  of  the  simulation  will  be  for  a  single  night  of  two  two-hour  periods  to  simulate  the  available 
collection  periods  due  to  space  objects  not  able  to  be  viewed  using  an  optical  telescope  if  the  object  falls 
in  the  shadow  of  the  Earth.  All  activities  use  a  normal  distribution  to  represent  when  satellites  are 
generated  or  accomplish  an  activity  until  data  collection  has  been  performed  to  more  accurately  represent 
the  physical  realities  for  the  system.  Queue  blocks  use  a  timer  to  release  items  after  four  minutes  to 
represent  the  object  is  constantly  moving  in  the  system  and  therefore  able  to  be  a  missed  opportunity  to 
collect  on  the  target. 

3.4.  Data  Collection 

A  large  part  of  the  any  simulation  is  in  data  collection  and  if  troubles  did  not  arise  within  the  model 
translation  part  of  the  process,  additional  data  collection  would  have  resulted.  Preliminary  data  collection 
comes  from  screen  captures  of  data  showing  the  expected  upcoming  objects  as  illustrated  in  Figure  1. 
This  representative  screenshot  is  typical  of  number  of  objects  in  view  at  any  given  time  along  with  the 
amount  of  time  available  to  be  able  to  view  the  object. 

The  slew  time  activity  for  the  scope  was  represented  by  an  Equation(l)  block  knowing  the  slew  speed 
from  the  telescope  manual  and  the  last  position  of  the  telescope. 

Therefore,  using  the  screenshots  again,  a  determination  was  made  that  using  a  normal  distribution 
would  suffice  for  now  with  a  mean  of  seven  minutes  and  a  standard  deviation  of  two  minutes. 


Figure  1:  Satellite/Space  Object  Tracking  Screenshot 
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The  values  and  concepts  were  then  used  to  develop  the  simulation  model  using  EXTENDSIM  8™. 
Difficulties  arose  dealing  with  the  pre-empt  function  of  the  activity  block  in  order  to  simulate  the  priority 
of  what  object  is  to  be  tracked  in  the  system.  The  authors  discovered  and  confirmed  with  technical 
support  there  is  a  computer  code  problem  with  the  software.  They  did  make  a  suggestion  on  a  possible 
fix  with  an  Equation(l)  block  to  force  a  timing  check  for  the  preempt  signal.  Developing  code  to  make 
the  proposed  change  has  made  the  system  disregard  any  higher  priority  items  entering  the  modeling  for 
the  telescopes.  Also,  issues  with  resource  pools  apparently  allowing  more  resources  then  what  were 
allowed  was  quite  troublesome,  but  a  solution  was  finally  found  using  gates  down  the  separate  paths  for 
the  telescopes. 

3.6.  Verified? 

At  this  point  in  time,  the  model  is  failing  verification,  because  tests  showed  increasing  numbers  of 
higher  priority  objects  entering  theTeleTrak  system.  This  however,  allowed  for  identification  of  issues 
with  items  of  the  same  priority  causing  software  errors.  A  good  use  of  DOE  could  be  used  at  this  step  to 
develop  test  scenarios  on  different  functions  as  they  are  built  into  the  system  and  used  again  when  another 
change  occurs  to  the  model . 

3.7.  Validated? 

With  the  model  not  getting  out  of  the  verification  step  it  is  difficult  to  validate  the  model.  However,  it 
has  been  demonstrated  that  queue  blocks  just  using  a  timer  to  determine  if  an  object  should  just  exit  the 
system  is  not  the  best  solution.  A  method  to  enhance  validity  in  this  area  is  to  develop  an  Equation  Queue 
block  factor  in  the  stamped  view  time  attribute  for  a  specific  item  and  push  the  item  out  if  it  meets  a  to  be 
determined  threshold. 

3.8.  Experimental  Design 

Again  since  a  working  model  of  the  system  does  not  exist  it  can  only  be  discussed  in  terms  of  the 
proposed  Experiment  Design.  Statistics  show  that  for  thirty  runs  of  a  test  in  an  unknown  population  a 
normal  distribution  curve  to  model  the  data  could  be  used.  Since  the  model  takes  a  relatively  short 
amount  of  time  to  run,  thirty  runs  of  the  simulation  would  be  run  to  collect  data  on  slew  time  with  the 
telescopes  free  to  move  from  their  last  location.  This  data  would  then  be  analyzed  to  determine  which 
distribution  (normal,  lognormal,  beta)  to  use  on  where  to  "home"  telescopes  before  the  next  collection. 
Thirty  runs  of  the  simulation  would  then  be  run  using  the  home  location  to  slew  from  and  the  analysis 
compared.  The  expected  results  are  a  longer  time  to  collect  on  target,  but  would  also  likely  decrease  the 
number  of  objects  tracked  over  the  course  of  the  night. 

4.  During/Post-Experimental  Phase 

The  final  steps  (Production  Runs  and  Analysis,  More  Runs?,  Documentation  and  Reporting,  and 
Implementation)  were  not  examined  in  depth,  because  of  unsuitability  of  the  simulation  model  in  its 
current  form.  If  implemented  the  results  of  the  analysis  would  be  presented  during  SPARK  and  T eleT rak 
team  meetings  to  discuss  implications  of  the  system,  and  state  requirements  for  the  system  beyond  "do  the 
best  you  can  with  what  you  can  get." 


158 


Michael  W.  Schreiner  and  Joseph  R.  Wirthlin  /  Procedia  Computer  Science  8  (2012)  153  -  158 


5.  Conclusions 

The  main  objectives  of  developing  a  working  simulation  that  would  model  the  "to-be"  architecture  and 
optimal  starting  location  for  the  telescope  were  not  realized.  However,  the  goal  of  learning  design  of 
experiments  and  simulation  modeling  were  achieved.  The  twelve  step  simulation  and  seven  step  DOE 
processes  provide  a  firm  foundation  for  further  work  within  these  important  realms.  Key  points  as  a 
takeaway  from  what  has  been  accomplished  so  far  are: 

(1)  Although  the  project  plan  as  scheduled  allowed  for  achieving  stated  goals,  additional  time  must  be 
allowed  to  learn  how  to  use  a  simulation  tool  if  advanced  modeling  behavior  is  required. 

(2)  Use  an  incremental  approach  in  simulation  as  it  is  done  in  software,  by  only  doing  small  portions  at 
a  time  then  testing  to  see  if  the  expected  model  works. 

(3)  When  designing  for  experiments,  ensure  that  a  sampling  of  representative  cases  is  used  to  test  a 
simulation  or  in  collecting  data  for  the  model.  A  test  case  for  the  simulation  involved  having  conditions 
within  the  system  related  to  what  priority  was  in  the  system.  In  collecting  a  sample  for  slew  time,  it  was 
seen  that  the  system  would  alternate  slew  directions,  so  even  if  the  next  target  was  a  few  degrees  azimuth 
to  the  right  and  it  was  just  collecting  on  an  object  while  slewing  to  the  right,  the  telescopes  would  take  the 
time  to  spin  nearly  360  degrees  before  tracking  the  object  before  it  started  tracking  the  object. 

(4)  M  ove  to  another  aspect  of  the  model  if  a  particular  process  or  logic  is  not  working  in  the  model. 
Seek  help  early  and  from  experts  when  these  circumstances  occur.  The  help  will  eventually  arrive  or 
another  solution  might  reveal  itself,  but  at  least  another  portion  of  the  model  will  be  completed. 

(5)  Use  a  simulation  when  variables  are  unknown,  and  understanding  of  the  system  is  only  partly 
understood.  If  a  system  can  be  represented  in  a  logical  model  (equations),  use  that  method  to  describe  the 
behavior. 

(6)  When  using  a  simulation,  particularly  discrete  event,  the  timing  of  when  an  event  occurs  is  crucial. 
Depending  on  how  an  object  is  connected  and  when  it  was  created  might  force  the  timing  of  the 
calculation  to  occur  too  late  in  the  process. 
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